What is claimed is: 



1 L Apparatus for controlling a rate at which a bit stream encoded at a variable rate is 

2 output, 

3 the apparatus comprising: 

4 a queue for receiving and storing the bit stream; 

5 a processor for applying a model of a receiver for the bit stream to information read 

6 from the bit stream to determine an output rate for the bit stream; and 

7 output apparatus responsive to the output rate for outputting the bit stream from the 

8 queue. 

1 2. The apparatus set forth in claim 1 wherein: 

2 the processor determines the output rate such that the receiver of the model will 

3 neither overflow nor underflow. 

1 3. The apparatus set forth in claim 2 wherein: 

2 the processor is further responsive to a state of the queue and further determines the 

3 output rate such that the queue will not overflow. 

1 4. The apparatus set forth in claim 3 wherein: 

2 the processor further determines the output rate such that the queue will not 

3 underflow. 

1 5. The apparatus set forth in any of claims 1 through 4 wherein: 

2 the bit stream is a sequence of components, the components having varying lengths 

3 and each component including timing information indicating when the receiver must 

4 process the component and 

5 the model includes 

6 a current set of components which is the set of the components in the queue 

7 together with the set of the components that have been sent to but not yet processed by 
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8 the receiver, 

9 the timing information for the components in the current set, 

10 the sizes of the components in the current set, and 

1 1 a size of a bit buffer in the receiver of the model. 

1 6. The apparatus set forth in claim 5 wherein: 

2 the components are digitally-encoded video images. 

1 7. The apparatus set forth in claim 6 wherein: 

2 the video images are encoded according to the MPEG-2 standard. 

1 8. The apparatus set forth in claim 2 wherein: 

2 the bit stream is a sequence of components, the components having varying lengths 

3 and each component including timing information indicating when the receiver must 

4 process the component; 

5 the model includes 

6 a current set of components which is the set of the components in the queue 

7 together with the set of the components that have been sent to but not yet processed by 

8 the receiver, 

9 the timing information for the components in the current set, 

10 the sizes of the components in the current set, and 

11 a size of a bit buffer in the receiver of the model; and 

12 the processor determines the output rate for a given period of time by determining a 

13 minimum rate such that each component is output from the queue before the receiver 

14 must process the component and a maximum rate such that the total size of the 

1 5 components in the set of the components that have been sent to but not yet processed by 

1 6 the receiver does not exceed the size of the bit buffer in the receiver of the model. 
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9. The apparatus set forth in claim 8 wherein: 

the processor increases the minimum rate for the given time period according to the 
number of bits that remain in the queue from a current component that is currently being 
output from the queue and the amount of time remaining until the current component 
must have been output from the queue and 

the processor decreases the maximum rate for the given time period as the total size 
of the components in the set of components that have been sent to the receiver and not 
yet processed approaches the size of the bit buffer in the receiver of the model. 

10. A method of dynamically determining as output rate at which a bit stream encoded 
at a variable rate is output to a receiver, 

the method comprising the steps of; 

receiving and storing the bit stream in a queue; 

in a processor, applying a model of the receiver to information read from the bit 
stream to determine the output rate; and 

outputting the bit stream from the queue at the output rate. 

11. The method set forth in claim 10 wherein: 

the output rate is determined such that the receiver of the model will neither overflow 
nor underflow. 

12. The method set forth in claim 1 1 wherein the method further comprises the step of: 
determining a state of the queue; and 

the step of determining the output rate further determines the output rate such that 
the queue will not overflow. 

13. The method set forth in claim 12 wherein: 

the step of determining the output rate further determines the output rate such that 
the queue will not underflow. 
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1 14. The method set forth in any of claims 10 through 13 wherein 

2 the bit stream is a sequence of components, the components having varying lengths 

3 and each component including timing information indicating when the receiver must 

4 process the component, 

5 the model includes 

6 a current set of components which is the set of the components in the queue 

7 together with the set of the components that have been sent to but not yet processed by 

8 the receiver, 

9 the timing information for the components in the current set, 

10 the sizes of the components in the current set, and 

11 a size of a bit buffer in the receiver of the model, and 
12 

13 the step of determining the output rate is done for a given period of time and comprises 

14 the steps of 

1 5 determining a minimum rate such that the component is output from the queue before 

16 the receiver must process the component and 

17 determining a maximum rate such that the total size of the components in the set of 

18 the components that have been sent to but not yet processed by the receiver does not 

19 exceed the size of the bit buffer in the receiver of the model. 

1 15. The method set forth in claim 14 wherein: 

2 the step of determining the minimum rate increases the minimum rate for the given 

3 time period according to the number of bits that remain in the queue from a current 

4 component that is currently being output from the queue and the amount of time 

5 remaining until the current component must have been output from the queue and 

6 the step of determining the maximum rate decreases the maximum rate for the given 

7 time period as the total size of the components in the set of components that have been 

8 sent to the receiver and not yet processed approaches the size of the bit buffer in the 
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9 receiver of the model. 

1 16. The method set forth in claim 14 wherein: 

2 the components are digitally-encoded video images. 

1 17. The method set forth in claim 16 wherein: 

2 the video images are encoded according to the MPEG-2 standard. 

1 18. A multiplexer for multiplexing a plurality of variable-rate bit streams onto a medium, 

2 the multiplexer comprising: 

3 a receiver for receiving the bit streams; 

4 a transmitter coupled to the receiver for transmitting the bit streams on the medium, 

5 each bit stream receiving a dynamically-variable portion of the bandwidth of the medium; 

6 and 

7 a bandwidth portion controller coupled between the transmitter and the receiver for 

8 dynamically determWng the variable portion for each bit stream using an output rate that 

9 is obtained by applying a model of a receiver for the bit stream to information read from 

10 the bit stream. 

1 19. The multiplexer set forth in claim 1 8 wherein: 

2 the bandwidth portion controller determines the output rate for each bit stream such 

3 that the receiver for the bit stream will neither overflow nor underflow. 

1 20. The multiplexer set forth in claim 19 wherein: 

2 the bandwidth portion controller further comprises a queue for each bit stream, the 

3 queue being coupled between the receiver and the transmitter and serving to receive the 

4 bit stream from the receiver and store the bit stream for output by the transmitter; and 

5 the bandwidth portion controller is further responsive to a state of the queue for each 

6 bit stream and further determines the output rate of the bit stream such that the bit 
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stream's queue will not overflow. 



8 21. The multiplexer set forth in claim 20 wherein: 

9 the bandwidth portion controller further determines the output rate for each bit stream 
10 such that the bit stream's queue will not underflow. 

1 22. The multiplexer set forth in any of claims 18 through 21 wherein: 

2 the bit stream is a sequence of components, the components having varying lengths 

3 and each component including timing information indicating when the receiver must 

4 process the component and 

5 the model includes 

6 a current set of components which is the set of the components in the queue 

7 together with the set of the components that have been sent to but not yet processed by 

8 the receiver, 

9 the timing information for the components in the current set, 

10 the sizes of the components in the current set, and 

1 1 a size of a bit buffer in the receiver of the model. 

1 23. The multiplexer set forth in claim 22 wherein: 

2 the components are digitally-encoded video images. 

1 24. The multiplexer set forth in claim 23 wherein: 

2 the video images are encoded according to the MPEG-2 standard. 

1 25. The multiplexer set forth in claim 20 wherein: 

2 the bit stream is a sequence of components, the components having varying lengths 

3 and each component including timing information indicating when the receiver must 

4 process the component; 

5 the model includes 
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6 a current set of components which is the set of the components in the queue 

7 together with the set of the components that have been sent to but not yet processed by 

8 the receiver, 

9 the timing information for the components in the current set, 

I o the sizes of the components in the current set, and 

I I a size of a bit buffer in the receiver of the model; and 

12 the bandwidth portion controller determines the output rate for a given period of time 

1 3 by detennining a minimum rate such that each component is output from the queue before 

14 the receiver must process the component and a maximum rate such that the total size of 

15 the components in the set of the components that have been sent to but not yet processed 

1 6 by the receiver does not exceed the size of the bit buffer in the receiver of the model. 

1 26. The multiplexer set forth in claim 25 wherein: 

2 the bandwidth portion controller increases the minimum rate for the given time period 

3 according to the number of bits that remain in the queue from a current component that 

4 is currently being output from the queue and the amount of time remaining until the 

5 current component must have been output from the queue and 

6 the bandwidth portion controller decreases the maximum rate for the given time 

7 period as the total size of the components in the set of components that have been sent 

8 to the receiver and not yet processed approaches the size of the bit buffer in the receiver 

9 of the model. 

1 27. The multiplexer set forth in any of claims 1 8, 19, or 20 wherein: 

2 Each bit stream has a priority; and 

3 the bandwidth portion controller further determines the variable portion for each bit 

4 stream in accordance with the bit stream's priority. 

1 28. The multiplexer set forth in any of claims 25 or 26 wherein: 

2 each bit stream has one of a plurality of priorities; 
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3 the plurality of priorities includes a first priority according to which each component 

4 of the bit stream having the priority will be received in the bit stream' s receiver at the time 

5 specified for the component; and 

6 where necessary to give a bit stream with the first priority a rate such that each 

7 component will be received in the bit stream's receiver at the time specified for the 

8 component, the bandwidth portion controller reduces the variable portion for at least one 

9 other bit stream which does not have the first priority. 

1 29. The multiplexer set forth in claim 28 wherein: 

2 the plurality of priorities includes another priority according to which the bit stream's 
y3 3 receiver will never underflow; and 

^ 4 where necessary to give a bit stream with the first priority a rate such that each 

,p 5 component will be received in the bit stream's receiver at the time specified for the 

3 6 component, the bandwidth portion controller reduces the variable portion for at least one 

y - 7 of the bit streams having the other priority to that required to receive at least a minimum 

O 8 component such that underflow will not occur. 

S{ 1 30. The multiplexer set forth in claim 28 wherein: 

H 2 the plurality of priorities includes another priority according to which the bit stream's 

3 receiver may underflow; and 

4 where necessary to give a bit stream with the first priority a rate such that each 

5 component will be received in the bit stream's receiver at the time specified for the 

6 component, the bandwidth portion controller reduces the variable portion for at least one 

7 of the bit streams having the other priority to nothing. 

1 31. The multiplexer set forth in any one of claims 18 through 21 wherein: 

2 the bandwidth portion controller allocates an output rate for each bit stream that is 

3 between a minimum rate and a maximum rate obtained by applying the model to the 

4 information. 



39 




1 32. The multiplexer set forth in claim 3 1 wherein: 

2 The bandwidth portion controller allocates the minimum rate for each bit stream and 

3 if bandwidth remains after that is done, allocates a higher rate up to the maximum rate. 

1 33. The multiplexer set forth in claim 32 wherein: 

2 the bandwidth portion controller allocates the higher rate for each channel in 

3 proportion to the difference between the maximum and minimum bit rates for the channel. 

1 34. Improved apparatus for providing a plurality of variable-rate bit streams to a medium, 
3 2 the apparatus comprising 

,TI 3 a plurality of encoders for receiving constant-rate bit streams and producing variable- 
's 4 rate bit streams therefrom and 

§ 5 a transmitter for outputting the bit streams to the medium 

;n 6 and the apparatus having the improvement comprising: 

h 7 the multiplexer set forth in any of claims 1 8 through 21, the multiplexer being coupled 

^ 8 between the encoders and the transmitter. 

fl 1 35, A method of multiplexing a plurality of variable-rate bit streams onto a medium, 

2 the method comprising the steps of: 

3 receiving the bit streams; 

4 for each bit stream, dynamically obtaining an output rate by applying a model of a 

5 receiver for the bit stream to information read from the bit stream; 

6 for each bit stream, using the output rate determined for the bit stream to dynamically 

7 determine a variable portion of the bandwidth of the medium; and 

8 . for each bit stream, outputting the bit stream to the medium using the bit stream's 

9 variable portion of the bandwidth. 
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10 36. The method set forth in claim 35 wherein: 

1 1 the output rate is determined such that the receiver of the model will neither overflow 

12 nor underflow. 

1 37. The method set forth in claim 36 wherein the method further comprises the step of: 

2 determining a state of the queue; and 

3 the step of determining the output rate further detennines the output rate such that 

4 the queue will not overflow, 

1 38. The method set forth in claim 37 wherein: 

2 the step of determining the output rate further determines the output rate such that 

3 the queue will not underflow. 

1 39, The method set forth in any of claims 35 through 38 wherein 

2 the bit stream is a sequence of components, the components having varying lengths 

3 and each component including timing information indicating when the receiver must 

4 process the component, 

5 the model includes 

6 a current set of components which is the set of the components in the queue 

7 together with the set of the components that have been sent to but not yet processed by 

8 the receiver, 

9 the timing information for the components in the current set, 

10 the sizes of the components in the current set, and 

1 1 a size of a bit buffer in the receiver of the model, and 

12 the step of determining the output rate is done for a given period of time and comprises 

13 the steps of 

1 4 detennining a minimum rate such that the component is output from the queue before 

15 the receiver must process the component and 

1 6 determining a maximum rate such that the total size of the components in the set of 
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17 the components that have been sent to but not yet processed by the receiver does not 

1 8 exceed the size of the bit buffer in the receiver of the model, 

19 40. The method set forth in claim 39 wherein: 

20 the step of determining the minimum rate increases the minimum rate for the given 

21 time period according to the number of bits that remain in the queue from a current 

22 component that is currently being output from the queue and the amount of time 

23 remaining until the current component must have been output from the queue and 

24 the step of determining the maximum rate decreases the maximum rate for the given 

25 time period as the total size of the components in the set of components that have been 

26 sent to the receiver and not yet processed approaches the size of the bit buffer in the 

27 receiver of the model. 

1 41. The method set forth in claim 39 wherein: 

2 the components are digitally-encoded video images. 

1 42, The method set forth in claim 41 wherein: 

2 the video images are encoded according to the MPEG-2 standard. 

1 43. The method set forth in any of claims 35, 36, or 37 wherein: 

2 Each bit stream has a priority; and 

3 the step of determining the variable portion does so in accordance with the bit 

4 stream's priority. 

1 44. The method set forth in claim 39 wherein: 

2 each bit stream has one of a plurality of priorities; 

3 the plurality of priorities includes a first priority according to which each component 

4 of the bit stream having the priority will be received in the bit stream's receiver at the time 

5 specified for the component; and 
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6 where necessary to give a bit stream with the first priority a rate such that each 

7 component will be received in the bit stream's receiver at the time specified for the 

8 component, the step of dynamically determining the variable portion reduces the variable 

9 portion for at least one other bit stream which does not have the first priority. 

1 45. The method set forth in claim 44 wherein: 

2 the plurality of priorities includes another priority according to which the bit stream's 

3 receiver will never underflow; and 

4 where necessary to give a bit stream with the first priority a rate such that each 

5 component will be received in the bit stream's receiver at the time specified for the 

6 component, the step of dynamically determining the variable portion reduces the variable 

7 portion for at least one of the bit streams having the other priority to that required to 

8 receive at least a minimum component such that underflow will not occur. 

1 46. The method set forth in claim 44 wherein: 

2 the plurality of priorities includes another priority according to which the bit stream's 

3 receiver may underflow; and 

4 where necessary to give a bit stream with the first priority a rate such that each 

5 component will be received in the bit stream's receiver at the time specified for the 

6 component, the bandwidth portion controller reduces the variable portion for at least one 

7 of the bit streams having the other priority to nothing. 

1 47. The method set forth in any one of claims 36 through 38 wherein: 

2 the step of obtaining the output rate obtains a minimum rate and a maximum rate. 

1 48. The method set forth in claim 47 wherein: 

2 the step of determining a variable portion allocates the minimum rate for each bit 

3 stream as the variable portion and if bandwidth remains after that is done, allocates a 

4 higher rate up to the maximum rate. 
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1 49. The multiplexer set forth in claim 48 wherein: 

2 the bandwidth portion controller allocates the higher rate for each channel in 

3 proportion to the difference between the maximum and minimum bit rates for the channel. 

1 50, A multiplexer for multiplexing a plurality of MPEG-2 bit streams onto a transmission 

2 medium, each bit stream containing a plurality of varying-length pictures representing 

3 video images, each picture containing timing information specifying at least a decoding 

4 time for the picture, and each bit stream being provided via the transmission medium to 

5 a decoder, 

6 the multiplexer comprising: 

7 a receiver for receiving the bit streams; 

8 a transmitter for providing the bit streams to the transmission medium; and 

9 for each bit stream, 

10 a timing information detector for detecting the timing information in a picture 

1 1 being received by the receiver, 

12 a picture size detector for detecting the size of the picture being received by the 

13 receiver, and 

14 a transmission rate controller which is coupled between the receiver and the 

1 5 transmitter, the transmission rate controller providing the detected timing information and 

16 the detected picture size to a model of the bit stream's decoder and using the model to 

17 determine a portion of the received bit stream to be provided to the transmitter, the 

1 8 portion being sized to prevent overflow of the decoder and to ensure that the picture will 

19 be received in the decoder prior to the decoding time. 

1 51. The apparatus set forth in claim 50 wherein the transmission rate controller 

2 comprises: 

3 a queue into which the bit stream is written from the receiver and from which the 

4 portions of the bit stream are read to the transmitter, and 

5 a queue fullness detector for detecting fullness of the queue; and 
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6 the transmission rate controller is further responsive to the queue fullness detector to 

7 determine the size of the portion to be transmitted such that the queue does not overflow. 

1 52, The multiplexer set forth in claim 5 1 wherein: 

2 the model of the decoder apparatus includes 

3 a current set of pictures which is the set of the pictures in the queue together with 

4 the set of the pictures that have been sent to but not yet processed by the receiver, 

5 the timing information for the pictures in the current set, 

6 the sizes of the pictures in the current set, and 

7 a size of the decoder and 

8 the transmission rate controller determines the output rate for a given period of time by 

9 determining a minimum rate such that each component is output from the queue before 

1 0 the decoder must process the component and a maximum rate such that the total size of 

1 1 the components in the set of the components that have been sent to but not yet processed 

12 by the decoder does not exceed the size of the decoder. 

1 53. The multiplexer set forth in claim 52 wherein: 

2 the transmission rate controller increases the minimum rate for the given time period 

3 according to the number of bits that remain in the queue from a current component that 

4 is currently being output from the queue and the amount of time remaining until the 

5 current component must have been output from the queue and 

6 the transmission rate controller decreases the maximum rate for the given time period 

7 as the total size of the components in the set of components that have been sent to the 

8 decoder and not yet processed approaches the size of the decoder. 

1 54. The multiplexer set forth in any of claims 50 through 53 further comprising: 

2 a transmission medium bandwidth allocator coupled to the transmission rate 

3 controllers, 

4 the bandwidth allocator being responsive to the transmission controllers to allocate 
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5 the bandwidth of the transmission medium among the transmission rate controllers such 

6 that at least the minimum portion of each bit stream is transmitted and the transmission 

7 rate controllers being responsive to the bandwidth allocator to transmit the portion as 

8 determined by the bandwidth allocator. 

1 55, The multiplexer set forth in claim 54 wherein: 

2 the transmission rate controller sets the output rate for each bit stream to a rate which 

3 is between a minimum rate and a maximum rate obtained by applying the model to the 

4 information. 

1 56. The multiplexer set forth in claim 55 wherein: 

2 The bandwidth allocator allocates the minimum rate for each bit stream and if 

3 bandwidth remains after that is done, allocates a higher rate up to the maximum rate for 

4 each bit stream. 

1 57. The multiplexer set forth in claim 53 wherein: 

2 the bandwidth allocator allocates the higher rate for each channel in proportion to the 

3 difference between the maximum and minimum bit rates for the channel. 

1 58. The multiplexer set forth in claim 54 wherein: 

2 each bit stream has one of a plurality of priorities; 

3 the plurality of priorities includes a first priority according to which each picture of 

4 the bit stream having the priority will be received in the bit stream's decoder at the time 

5 specified in the picture's time information; and 

6 where necessary to give a bit stream with the first priority a rate such that each picture 

7 will be received in the bit stream's decoder at the time specified for the picture, the 

8 bandwidth allocator reduces the variable portion for at least one other bit stream which 

9 does not have the first priority. 
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1 59. The multiplexer set forth in claim 58 wherein; 

2 the plurality of priorities includes another priority according to which the bit stream's 

3 decoder will never underflow; and 

4 where necessary to give a bit stream with the first priority a rate such that each picture 

5 will be received in the bit stream's decoder at the time specified for the picture, the 

6 bandwidth allocator reduces the variable portion for at least one of the bit streams having 

7 the other priority to that required to receive at least a minimum picture such that 

8 underflow will not occur. 

1 60. The multiplexer set forth in claim 58 wherein: 

2 the plurality of priorities includes another priority according to which the bit stream's 

3 decoder may underflow; and 

4 where necessary to give a bit stream with the first priority a rate such that each picture 

5 will be received in the bit stream's decoder at the time specified for the picture, the 

6 bandwidth allocator reduces the variable portion for at least one of the bit streams having 

7 the other priority to nothing. 

1 61. A method used in a multiplexer that multiplexes a plurality of variable-rate bit streams 

2 onto a medium for allocating the medium's bandwidth among the bit streams for a period 

3 of time, each bit stream having upper and lower bounds for the bit stream's bit rate during 

4 the period of time and 

5 the method comprising the steps of: 

6 for each bit stream, allocating bandwidth from the medium as required for the lower 

7 bound of the bit stream's rate during the period of time; 

8 determining how much bandwidth remains in the medium for the period of time; and 

9 allocating the remaining bandwidth among the variable-rate bit streams such that for 

1 0 each bit stream, the bit stream's bit rate during the period of time is no more than the bit 

1 1 stream' s upper bound. 
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1 62. The method set forth in claim 61 wherein the step of allocating the remaining 

2 bandwidth further comprises the steps of: 

3 for each bit stream, determining the difference between the upper and lower bounds 

4 for the bit stream; and 

5 for each bit stream, allocating the bit stream's portion of the remaining bandwidth in 

6 proportion to the difference, 

1 63. The method set forth in claim 62 wherein the step of allocating the bit stream's 

2 portion of the remaining bandwidth includes the step of allocating bit stream /' s portion 

3 AR(i) according to the formula 

(0-iLiiO Be 

Ajy/)= — 

4 where B c is the residual bandwidth for the period of time and T c is the period of time. 

1 64. A multiplexer for multiplexing a plurality of variable-rate bit streams onto a medium, 

2 the multiplexer being characterized in that: 

3 the multiplexer allocates bandwidth belonging to the medium to the variable-rate bit 

4 streams according to any of the methods set forth in claims 6 1 through 63 . 

1 65. Apparatus for providing a plurality of variable-rate bit streams to a medium, 

2 the apparatus comprising 

3 a plurality of encoders for receiving constant-rate bit streams and producing variable- 

4 rate bit streams therefrom, 

5 a multiplexer for receiving the variable-rate bit streams from the encoders, and 

6 a transmitter for receiving the variable-rate bit streams and outputting the variabie- 

7 rate bit streams to the medium, 

8 the apparatus being characterized in that 
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9 the multiplexer is a multiplexer according to claim 64. 

1 66. The method set forth in any of claims 61 through 63 wherein: 

2 at least one of the bit streams in the plurality thereof is a bit stream produced by 

3 encoding a sequence of video images according to the MPEG-2 standard. 

1 67. A multiplexer for multiplexing a plurality of variable-rate bit streams produced by 

2 encoders onto a medium, 

3 the multiplexer comprising: 

4 a receiver for receiving the bit streams; 

5 a transmitter coupled to the receiver for transmitting the bit streams on the medium, 

6 each bit stream receiving a portion of the bandwidth of the medium; and 

7 a bandwidth portion controller coupled between the receiver and the transmitter for 

8 dynamically controlling the rate at which at least one of the encoders encodes the 

9 bitstream produced by the encoder, the bandwidth portion controller using rate 

10 information that is determined by applying a model of a receiver for the bit stream to 

11 information read from the bit stream. 

1 68. Apparatus for controlling a rate at which an encoder encodes a bit stream, 

2 the apparatus comprising: 

3 a queue for receiving and storing the bit stream from the encoder; and 

4 a processor coupled to the bit stream and to the encoder for applying a model of a 

5 receiver for the bit stream to information read from the bit stream to determine the rate 

6 at which the encoder encodes the bit stream. 
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